Repository navigation
[MRG] Fix dmmot_monge_1dgrid_loss raising on backward - #923
Open
xihaian251 wants to merge 3 commits into
Open
xihaian251 wants to merge 3 commits into
xihaian251 wants to merge 3 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #923 +/- ##
=======================================
Coverage 97.07% 97.07%
=======================================
Files 130 130
Lines 27095 27113 +18
=======================================
+ Hits 26302 26320 +18
Misses 793 793 🚀 New features to boost your workflow:
|
3 of 5 tasks
# Conflicts: # RELEASES.md
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Types of changes
Component:
ot.lp.dmmot. No public API change, no new dependency, no backend is modified.Motivation and context / Related issue
ot.lp.dmmot_monge_1dgrid_lossis meant to be differentiable — the call site is evencommented "define forward/backward relations for pytorch" — but calling backward on it
raises on every backend that supports autodiff.
Minimal reproduction:
TypeError: unsupported operand type(s) for *: 'numpy.ndarray' and 'Tensor'ValueError: custom_gradient function expected to return 1 gradients, but returned 2 instead.The forward value is correct in both cases; only the backward pass fails.
Root cause. In
ot/lp/dmmot.py:(A_copy)and(dual)are not tuples — parentheses around a single name are onlygrouping. So
inputsis one tensor of shape(n, d), whilegradsis a Python list ofdNumPy arrays, each of lengthn.set_gradientsexpects one gradient per input, inthe backend's own array type. Instrumented at the call site:
Neither existing test in
test/test_dmmot.pycalls backward: one compares a forward valueafter
nx.to_numpy, the other checks optimizer output. As a result this line has neverbeen executed in a backward pass by the test suite.
No existing issue covers this; I did not open one to avoid duplicating traffic, but I am
happy to create one if maintainers prefer.
Implementation
dual[j]is the dual potential of marginalj, and the dual objective issum_j <A[:, j], dual[j]>, so the gradient with respect toAis the potentials stackedas columns:
np.stack(dual, axis=1)has shape(n, d), matchingAexactly.nx.from_numpy(..., type_as=A_copy)converts to the backend's array type and matches theinput's dtype and device, which the previous NumPy list did not.
(A_copy,)and(grad,)are genuine single-element tuples, so the input/gradientarity is unambiguous for every backend.
per-marginal constant, and on mass-preserving perturbations any such constant
contributes nothing to the directional derivative. I deliberately did not centre
them: independent per-column centring is not guaranteed to preserve the multi-marginal
dual constraints, so this PR keeps the change purely structural. The returned gradient
is exactly
log["dual"]stacked as columns.log["dual"]andlog["dual objective"]are unchanged.How has this been tested (if it applies)
New non-regression tests. 8 cases in
test/test_dmmot.py:test_dmmot_monge_1dgrid_loss_gradientsovercoeff ∈ {1.0, 7.0, -2.0, 0.0}×(n, d) ∈ {(4, 2), (6, 3)}. Each asserts the forward value is unchanged, the gradientdtype matches the input, and the gradient equals
coefftimes the stacked duals. Thenon-square
(6, 3)case matters because it makes the input count differ from the oldlist length, so an arity-only fix would not pass it.
coeff = 0.0and-2.0coverexternal scaling of the loss.
Negative control. The unpatched function was extracted verbatim from a clean checkout
of
masterand bound onto the patched tree, then the same test file was run:All 8 new cases fail on
master— includingcoeff = 1.0, since this is a hard crashrather than a scaling error — and all 6 pre-existing cases still pass.
Additional local checks (PyTorch). 4 input shapes
(4,2),(6,3),(5,5),(8,2)×{float32, float64}×coeff ∈ {1, 7, -2, 0}= 32 combinations, all withmax|grad - coeff * expected| = 0.000e+00, gradient dtype and device preserved in everycase. The forward objective and
log["dual"]are bit-identical to the unpatched baseline.Test suites (local, two separate environments, not combined):
test/test_dmmot.pytest/suitetest/test_dmmot.pyThe unpatched baseline in the same environment gives 3575 passed / 468 skipped / 6 xfailed,
so the whole delta is the 8 new cases;
skippedandxfailedare unchanged.Lint:
ruff 0.5.2(the version pinned in.pre-commit-config.yaml), all four hooks —check ot/,check --preview --select=NPY201 ot/,check --ignore=D103,D400,E402 test/,format --check ot/— all pass.git diff --checkclean.On CI:
linux-torchis the job that actually exercises the new tests — all 8 arelisted individually as
PASSEDin its log (job total 2805 passed, 66 skipped, 6 xfailed).In
linux-tfand the other non-torch jobs they areSKIPPED, as intended by theskipif(not torch)guard.Backend capabilities not covered by this PR, stated explicitly:
stacked duals exactly. However
TensorflowBackend.set_gradientsseparately returnsgradsunchanged and discards the upstream cotangent, so an external coefficient isstill not propagated: measured
grad(c * loss) == grad(loss)forc ∈ {1, 7, -2, 0}.That is an independent pre-existing defect in the backend, not addressed here. For
that reason no TensorFlow regression test with
c != 1is added — it could not passuntil the backend is fixed.
dmmot_monge_1dgrid_losscallsnx.to_numpy(A), so it is not traceable:jax.gradraisesTracerArrayConversionErrorboth before and after this change. ThisPR does not make the solver JAX-differentiable; that would require a separate
rewrite of the solver internals.
torch.compile, and Python versions other than 3.13.Local results are not a substitute for GitHub Actions; the checks on this branch are the
authoritative result.
PR checklist
Notes on the checklist: no public signature or docstring changes, so no documentation
update is required, and no
examples/script is needed since CONTRIBUTING asks for oneonly when adding functionality. The tests box reflects both the local runs above and the
passing CI jobs on this branch. The
RELEASES.mdentry is added under#### Closed issuesin the style of the neighbouring entries, and references this PR.